With the development of digital imaging devices, it has become possible to capture high-resolution still images at a relatively low cost. However, it remains difficult to capture moving images that smoothly move at high speed (for example, at a high frame rate of approximately 30 to 60 frames per second or a higher frame rate than this) with a resolution equivalent to that of the aforementioned high-resolution still images.
The reason for the difficulty is that since an enormous amount of image information is obtained per unit time in the case of a high-speed high-resolution image capturing, a transfer rate of image information becomes insufficient in an imaging device. The problem of the transfer rate can be solved by using an image capturing apparatus that employs a special imaging device whereby the image information can be transferred in parallel. However, such an image capturing apparatus is costly and also causes another problem that properties of the imaging device need to be adjusted.
As an example of a conventional image generation method of obtaining a high-speed high-resolution (namely, high spatiotemporal resolution) video sequence from a limited amount of video information, the following method is known. That is, there is an image generation method whereby morphing is performed through detecting corresponding points between a frame of a video sequence captured by a high-speed low-resolution camera and a frame of a video sequence captured by a low-speed high-resolution camera (see Patent Reference 1, for example).
FIG. 19 is a block diagram showing a configuration of a conventional image generation apparatus disclosed in Patent Reference 1. In FIG. 19, a high-speed low-resolution camera A01 and a low-speed high-resolution camera A02 are shooting the same subject in synchronization with each other at the same angle of view. With reference to FIG. 19, a conventional image generation method is described.
FIG. 20 is a diagram showing moving images obtained through the image capturing performed by the camera A01 and the camera A02. In FIG. 20, frame images respectively captured by the camera A01 and the camera A02 are shown in chronological order. A frame B01 and a frame B02 represent frame images captured by the camera A02, while frames B11 to B15 represent frame images captured by the camera A01.
Here, a difference in resolution between the frame images is shown by a difference in image size. As compared to the frames B01 and B02, the frames B11 to B15 have a smaller number of pixels and thus a lower resolution. However, the frame rate of the camera A01 for capturing images is higher than that of the camera A02. The camera A01 captures four frames while the camera A02 captures one frame.
Timings at which the camera A02 and the camera A01 capture respective frames synchronize with each other. For example, the frame B01 and the frame B11 are captured at the same time, and the frame B02 and the frame B15 are also captured at the same time.
The following is a description about an example of a method of generating an intermediate frame B23 which is a frame to be inserted in the low-speed high-resolution video sequence corresponding to the time at which the frame B13 is captured.
A primary matching unit A03 obtains a correspondence relation of pixels between adjacent frames in the high-speed low-resolution video sequence captured by the camera A01. Here, the correspondence relation of pixels refers to a relation between a pixel in one frame image and a pixel in another frame image, each of these pixels representing the same specific part of the subject.
By combining the correspondence relations of pixels between the adjacent frames, from the frame B11 to the frame B13 of the high-speed low-resolution video sequence, the primary matching unit A03 obtains the correspondence relation of pixels between the frame B11 and the frame B13. The time at which the frame B11 is captured corresponds to the time at which the frame B01 of the low-speed high-resolution video sequence is actually captured. Similarly, the time at which the frame B13 is captured corresponds to a time of the frame B23 that is to be inserted in the low-speed high-resolution video sequence.
Next, a secondary matching unit A04 combines: a positional relation of pixels between the frame B01 of the low-speed high-resolution video sequence and the frame B11 of the high-speed low-resolution video sequence that are captured at the same time; a positional relation of pixels between the frame B13 and the frame B23 which correspond to each other in terms of time; and the correspondence relation of pixels between the frame B11 and the frame B13, which has been obtained by the primary matching unit A03. As a result of this, the secondary matching unit A04 determines the correspondence relation of pixels between the frame B01 and the frame B23.
Next, an image generation unit A05 determines pixel values of the frame B23 using the pixel values of the frame B01 on the basis of the correspondence relation of pixels between the frame B01 and the frame B23, so as to generate the high-resolution intermediate frame B23.
According to this procedure, other intermediate frames are similarly generated. In consequence, a high-resolution high-speed video sequence is generated.
Likewise, Non-Patent Reference 1 and Non-Patent Reference 2 disclose how to generate a high-resolution intermediate frame image using a video sequence captured by a high-speed low-resolution camera and a video sequence captured by a low-speed high-resolution camera. To be more specific, motion estimation is performed on the frames of the video sequence captured by the high-speed low-resolution camera. Then, on the basis of the result of the motion estimation, morphing is performed on the video sequence captured by the low-speed high-resolution camera so as to generate the high-resolution intermediate frame image.
As described so far, according to the conventional technique, a high-speed high-resolution video sequence is obtained by generating high-resolution intermediate frame images through combining high-speed low-resolution moving images and low-speed high-resolution moving images and then inserting each generated intermediate frame image in the low-speed high-resolution video sequence. This is to say, the high-speed high-resolution video sequence can be generated using a smaller amount of image information than the actual amount of image information required to originally generate a high-speed high-resolution video sequence. This can ease the problem of the insufficient transfer rate of image information that is caused when high-speed high-resolution moving images are actually captured.    Patent Reference 1: Japanese Unexamined Patent Application Publication No. 2003-203237 (FIG. 18)    Non-Patent Reference 1: Toru MATSUNOBU, et al., “Generation of High Resolution Video Using Morphing”, Technical report of IEICE, PRMU 2004-178    Non-Patent Reference 2: Kiyotaka Watanabe, et al., “Generation of High Resolution Video Sequence from Two Video Sequences with Different Spatio-temporal Frequencies”, Information Technology Letters (FIT2004), Vol. 3, No. LI-004, 2004